Services in a service-oriented provider network can utilize distributed computing nodes to perform various functions required for the services. The provider network may include a load balancer to distribute service requests or service workloads. The load balancer may distribute the service requests to one or many of the distributed computing nodes in order to spread out the service requirements or demands on the computing nodes. A given computing node can have limits as to how many requests can be received and processed in a given period of time. When the given computing node reaches a maximum limit, future requests can be throttled and slowed down at the given computing node. The limit can cause additional service requests to return an error indicating that throttling has occurred. The additional service requests will be dropped by the given computing node.
While embodiments are described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that embodiments are not limited to the embodiments or drawings described. It should be understood, that the drawings and detailed description thereto are not intended to limit embodiments to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope as defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words “include,” “including,” and “includes” mean including, but not limited to. When used in the claims, the term “or” is used as an inclusive or and not as an exclusive or. For example, the phrase “at least one of x, y, or z” means any one of x, y, and z, as well as any combination thereof. As used throughout this application, the phrase “based on” a factor means based at least in part on the factor, rather than exclusively based on the factor, such that an event “based on” the factor can also be influenced or modified by another factor.